DOCUMENT RESUME 



ED 064 326 



TM 001 491 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



SDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Gall, Meredith D. ; And Others 

Comparison of Instructional Media in a Minicourse on 
Higher Cognitive Questioning. 

Apr 72 

18p.; Paper presented at the annual meeting of the 
American Educational Research Association (Chicago, 
Illinois, April 1972) 

MF-$0.65 HC~$3.29 

Behavior Change; Educational Research; Evaluation 
Techniques; Inquiry Training; ^Instructional Media; 
«Microteaching; Models; ^Teacher Behavior; «Teacher 
Education; ^Teaching Skills; Videotape Recordings 
♦Minicourse 9 



ABSTRACT 

The two main purposes of this study were: (1) to 

determine the effectiveness of a teacher training program «Minicourse 
9; Higher cognitive Questioning”; and (2) to compare the relative 
effectiveness of two instriictional techniques in changing teacher 
behavior-observation of videotaped model teachers displaying 
classroom skills, and reading of transcripts derived directly from 
the videotapes. A group of 54 teachers took Minicourse 9. A separate 
group of 24 teachers served as controls. A pre- and post-measure of 
each teacher’s discussion behavior were collected. The two student 
response measures— frequency and length of higher cognitive 
responses — both increased favorably as a consequence of the 
Minicourse. The study indicates that the course was ineffective in 
changing teachers' use of refocusing. It also indicates that the 
v^ritten version of Minicourse 9 is at least equally as effeccive as 
the video version. On some course indices, such as probing questions, 
the written version is clearly superior. However, the written 
materials used in this study were developed directly from videotapes, 
and as a result, probably had greater realism and interest than if 
composed by a writer who created the classroom dialogue. (DB) 
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Purpose of Study 

This study was conducted for two main purposes. First, the authors wished 
to determine the effectiveness of a teacher training program. Mini course 9: 
Higher Cognitive Questioning , in changing teacher and student behavior. 

The study was carried out as part of the research and developfiient cycle 
used at the Far West Laboratory to produce training materials (Borg et al, 
1970). A given set of materials such as Minicourse 9 is not released for 
general use until there is evidence indicating that it brings about signif- 
icant behavioral changes in the target audience. 

The second purpose of the study was to compare the relative effectiveness 
of two instructional techniques in changing teacher behavior. It is a 
well-established finding that a person can acquire new behavior patterns by 
observing these patterns displayed by another person (Bandura, 1965). This 
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learning process has been labeled "vicarious learning" or "modeling". 

In the present study, we were concerned with the question, "What is the 
relative effectiveness, as measured by changes in teachers' question- 
asking behavior, of two forms of modeling; (a) observation of videotaped 
model teachers displaying classroom skills; or (b) reading of transcripts 
derived directly from the videotapes (written modeling)?" Two previous 
studies have compared similar modeling treatments. Allen et al (1967) 
found no difference between video and symbolic modeling in training pre- 
service teachers to use higher cognitive questions. Koran (1969) found 
a significant, though small difference favoring video modeling for a 
similar skill and teacher population. 

/ 

i 

Method 

i 

1 

Treatments 

Mini courses . The Teacher Education Program cf the Far West Laboratory 
is developing a coordinated series of Mini courses, each covering a differ- 
ent set o^' basic teaching skills. Minicourses are designed primarily for 
inservice teachers, alwhough there have been some adaptations to preservice 
settings. Each Mini course is field tested at least three times before final 
release. The version tested here in two forms was developed for the second 
field test - the main one in the R & D cycle. ^ 

^ Since this study. Mini course 9 has completed the R & D cycle and is 
available in commercial form (Gall, Weathersby, and Dunning, 1971). 
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Minicourse 9 provides about 15 hours of training in four lessons. Each 
lesson follows the same sequence of: instruction in several teaching 
skills; a microteach session in which the teacher practices these skills 
with a small group of his students and videotapes It for later replay and 
self-analysis; and a reteach session, which is essentially another micro- 
teach session for further practice. 

Video Treatment . The instructional phase of this treatment consisted 
of a teacher handbook which explained the course skills and provided 
practice in discriminating and writing various types of higher cognitive 
questions. It also contained four instructional and four model videotapes 
which showed teachers using the skills in various classroom situations. 

Written Treatment. This treatment was the same as the video treatment 
except that trainees did not view any of the instructional or model video- 
tapes. Instead they read transcripts of these videotapes. The tran- 
scripts were derived from the shooting scripts of the videotapes. The 
narration was put in paragraph form, and the discussion clips were tran- 
scribed in the form of teacher-student interactions. 

Control Treatment . Trainees In this treatment received no instruction 
or visitation by Laboratory staff during the time that the video and 
written groups were taking Minicourse 9. The purpose of the control group 
was to determine the extent to which changes from pretesting to posttesting 
could be attributed to factors other than the experimental treatments. 
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Sampi e 

A total of 80 teachers In two California school districts were recruited 
on a voluntary basis to participate in the study. One group of 56 teachers 
was recruited to take Minicourse 9. These teachers were randomly assigned 
to the video and written versions of the course. Descriptive statistics 
on teachers' age, teaching experience, and grade level are given in 
lable 1. Eventually, two teachers dropped from the course, resulting In a 
final sample size of 54. 

A separate group of 24 teachers from the same school districts was recruited 
to serve as controls. These teachers were paid a stipend of $ 15.00 to 
conduct a pretape and posttape discussion lesson. As Table 1 shows, the 

/ 

control teachers differed from the treatment groups In generally being older 
and more experienced. 

V 

Data Collection Instruments -a 

To determine the relative effectiveness of the two treatment and one control 
conditions, a pre-measure and a post-measure of each teacher's discussion 
behavior were collected. 

One week prior to the start of the course, teachers In all three groups 
were assigned a one-page standard stimulus article on which they were to 
conduct a twenty-minute discussion with their classes. The articles were 
chosen so as to have the following characteristics: brevity; of Interest to 
students; at the students' reading level; and containing a mixture of fact 



o 

ERIC 

imiifiifftiiriaaij 



4 



5 



and thought-provoking material. All 4-6th grade teachers received one 
article; the junior high school teachers received another. 

The teachers were asked to have their students read the article a day prior 
to the audiotaped discussion, with no intervening discussion. One week 
after the course had concluded, teachers were given another standard stimu- 
lus article on which to base a discussion. Again the 4-6th grade teachers 
received one article; the junior high school teachers received another. 

An attempt was made to make the pre-post pairs of articles as equivalent 
as possible on the dimensions described above (brevity, interest, etc.). 

Because it is is difficult to obtain a good audiotape recording in a 

regular classroom setting, teachers were requested to use only one-half 

of their students for pre- and post-course taping. To select a random ^ 

sample of students, teachers referred to their class roster and selected 

the first, third, fifth student, etc. for inclusion in the sample. The 

same students were used for pre- and post-course taping. 

To facilitate later scoring of discussion behaviors, typed transcripts 
were made of all audiotapes. 

Scoring of Audiotapes . After the audiotapes had been transcribed, raters 
were trained to classify each question asked by a teacher into one of 
Bloom's six categories: Knowledge, Comprehension, Application, Analysis, 

Synthesis, and Evaluation. Two raters scored each transcript. Reliability 
coefficients ranged from .54 to .96; the median coefficient was .86. In 
the data analyses that follow, the question types except Knowledge were 
considered higher cognitive. 
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Following classification by type, raters judged whether each question 
"refocused" from the lesson topic to another lesson topic or to personal 
experience. The average interrater reliability coefficient was .71. 

As part of the data analysis, it was planned to compute the mean length 
and number of student responses to each type of higher cognitive question. 

Before this analysis could be undertaken, however, it was necessary to 
resolve all disagreements by raters in classification of each question. 

A third rater who had not been one of the two raters assigned to score 
a particular transcript was used to resolve the disagreements. In all 
instances in which one of the original raters had classified a teacher 
utterance as a question of some type, whereas the other original rater 
had not considered it to be a question, the third rater was instructed /• 

I 

to accept the scoring of the first-mentioned rater. 

j 

A research clerk counted the length of each student response to a question. 

However, if a student response was partially or entirely inaudible (as 
indicated by series of dots in the transcript), the response was not 
counted at all. Otherwise, the length of inaudible student responses 
would be unjustifiably shortened. However, inaudibility is not a biasing 
factor in computing the number of student responses to each type of ques- 
tion. Therefore, all student responses, irrespective of audibility, were 
entered into the computation of this variable. 
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Results 



Percentage of Higher Cognitive Questions 

Table 2 presents the mean percentage of higher cognitive questions asked 
before and after Mini course 9 by teachers in the video, written, and 
control groups. Analysis of covariance and follow-up comparison between 
adjusted means reveals that, at each grade level, the video and written 
groups significantly outperformed the control group; but the video and 
written groups did not differ significantly from each other. 

Frequency of use of each question type was also analysed, as a matter of 
interest. Results of these analyses indicated that percentage of higher 
cognitive questions increased for the video and written groups, primarily 
because of a sizable reduction in use of knowledge questions, with an 
accompanying small increase in use of higher cognitive questions. 

Number of Student Responses 

Table 3 presents the number of student answers given in response to 
teachers' higher cognitive questions. For a given teacher, the freque.cy 
of student higher cognitive answers is not necessarily equal to the fre- 
quency of teacher higher cognitive questions, since a given question 
might elicit more than a single response. This situation would occur if 
the teacher "redirected" the same question to several students, or if 
several students responded in turn to the same question. 
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Analyses of covariance, and subsequent comparison of adjusted means. In- 
dicated that the written group outperformed the controls at both grade 
levels. The video group outperformed the controls only at the 4-6th 
grade levels. Also, the written group significantly outperformed the video 

I 

group at the junior high school level; the same comparison at the 4-6th 
grade level approached significance at the .05 level. 

Length of Student Response 

One of the assumptions underlying Minlcourse 9 is that If teachers Increase 
their use of higher cognitive questions, more thoughtful student responses 
should occur. Furthermore, It is assumed that there Is a positive rela- 
tionship between thoughtfulness of a response and Its length. ^ 

1 

The mean number of words In student responses to higher cognitive questions 

I 

Is shown In Table 4. Only slight changes occurred in the treatment groups. 

Even these Increases can be attributed to factors other than the Minlcourse, ^ 

since the controls made comparable gains. 

The possibility occurred to the authors that only slight increases were 
observed because teachers In the treatment groups elicited more student 
responses on the posttape (see Table 3). Teachers might be eliciting more 
long responses, but If they were also eliciting more short responses, the 
mean response length would not Increase appreciably. Thus, the decision 
was made to determine whether there was an absolute Increase In frequency 
of long responses after training. A long response was operationally defined 
as a response 15 words or longer In length. Table 5 presents the results 
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of this analysis. For the elementary school sample, the number of long 
responses decreased slightly for the control group, whereas they in- 
creased by 27 percent in the video group, and by 86 percent in the written 
group. 

Less difference between the video and written groups was obtained in the 
junior high school sample. The increase for the video group was 38 per- 
cent, and for the written group it was 29 percent. The control group 
made only a slight percentage gain of 9 percent. 

The results of this analysis indicate that the video and written treat- 
ments did, in fact, lead teachers to elicit longer student responses in 
terms of absolute frequency of occurrence. The fact that the overall 
mean response length did not increase indicates that the treatments led 
teachers to elicH more short responses as well. 

Probing Questions 

Table 6 presents the mean number of probing questions asked by each group 
of teachers before and after Minicourse 9. Probing questions were narrowly 
defined as questions requiring the student to defend or justify his 
original response to a higher cognitive question; most probing questions 
were identified by the use of the question stem "Why?" Analyses of co- 
variance, and subsequent comparison of adjusted means, revealed that only 
the written group in the 4-6th grade sample made significant gains in use 
of this technique. 
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Refocusing Technique 

Table 7 presents the mean frequency with which each treatment grou;' used 
refocusing before and after Mini course 9. It is evident that only the 
video and written groups in the junior high school sample made gains, but 
even these did not exceed the gain made by the controls. Analyses of 
covariance supported this impression: there were no significant differ- 
ences between the three groups in either sample. 

Post-Course Questionnaire Data 

After posttaping was completed, each teacher in the video and written 
groups was administered a questionnaire to elicit his reactions to the 

/ 

course. Several of the items were designed to elicit teachers' percep- 
tions of the Minicourse's effectiveness. Distribution of responses to 
these items for each treatment v,roup is shown in Table 8. It appears ’ 

that teachers perceived Mini course 9 favorably, and that there were no ^4 

substantial differences between treatment groups in this respect. 



Conclusions 



The main objective of Mini course 9 is to increase teachers' use of higher 
cognitive questions in discussions. The data of the main ffleld test 
indicate that the course does indeed accomplish this objective. However, 
we may speculate whether the obtained increass^s rre of practical signifi- 
cance. For example, the junior high school virit^tr> group gained only 
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5.6 percentage points, f>^om 67.2 to 72.8 percent. However, It should be 
noted that the basel' .t: percentages were unusually high compared to those 
obtained in other studies (see Gall, 1970 for a review of these studies). 

Two explanations can be offered for this phenomenon: the teachers in the 
sample were unusually well trained ,rior to Minicourse 9; or the assigned 
topic caused teachers to conduct a much better discussion than if they 
were asked to select their own discussion topic. We have no reason to 
believe that the teachers were unusual. The second explanation seems much 
more plausible, especially since in all the studies reporting low baseline 
data, teachers were free to select their own discussion topics. 

Pursuing this line of reasoning, we may speculate that if Minicourse 9 
were evaluated with unassigned pretape and posttape topics, different per- 
centage gains would be obtained. Furthermore, since high baseline percent- 
ages are obtained using assigned topics, we may hypothesize that one of 
the things teachers learn in a training program on higher cognitive ques- 
tioning is to select lesson material that is conducive to a thoughtful 
discussion. 

The data on frequency of each question type indicate that teachers already 
have higher cognitive questioning skills prior to inservice training. The 
main change effected by the Mini course is to decrease teachers* use of 
Knowledge questions, thereby Increasing the percentage of higher cognitive 
questions. 

The two student response measures - frequency and length of higher cognitive 
responses - both increased favorably as a consequence of the Minicourse. 
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Although length of response Is probably rel'ted to Its quality, the authors 
realize that much better measures of this variable need to be developed.^ 

The study indicates that Minicourse 9 was ineffective in changing teachers' 
use of refocusing. Therefore, this technique was omitted from the next 
developmental version of the course. 

The study also indicates quite clearly that the written version of Mini- 
course 9 is at least equally as effective as the video version. On some 
course indices, such as probing questions, the written version is clearly 
superior. The reader should not construe this finding to mean that any. 
written materials are as effective as instructional videotapes. The 
written materials used in this study were developed directly from video- 

/ 

tapes; as a result, they probably had greater realism and interest than 
if they were composed by a writer relying only on imagination to create 
classroom dialogue. This finding influenced the design of the final, 
commercial version of the course: four of the instructional videotapes 
were made into films, in part because of their motivational appeal; the 
other four videotapes were converted to handbook form to lower cost of 
the product to the user. 



The final version of Minicourse 9 moves in this direction by presenting 
various criteria (e.g. clarity, complexity, defens ibility) which the 
teacher can use in evaluating his students' answers. 
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Table 1 

Descriptive Information about Participating Teachers 



Group 


Sample 

Size 


Age 


Years of Teaching 
Experience 


r ^ 


S.D. 


f 


S.D. 


I. 4.6 












Video 


16 


30.9 


8.7 


6.9 


8.1 


Written 


16 


36.5 


7.6 


9.4 


4.0 


Control 


13 


40.6 


10.6 


11.5 


10.9 


II. Junior High Sch 


ool 










Video 


10 


34.3 


7.6 


6.3 


3.6 


Wri tten 


12 


34.5 


7.5 


9.1 


6.5 


Control 


11 


42.5 


11.6 


14.7 


9.0 
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Table 2 

Percentage of Higher Cognitive Questions 



Group 


P 


recourse 


Postcours 


s 


X 


S.D. 


N 


X 


S.D. 


N 


I. 4-6th Grade Teach 


ers 












Video 


49.9 


13.2 


16 


67.0 


15.9 


16 


Wri tten 


40.4 


15.9 


16 


63.8 


15.4 


16 


Control 


52.2 


18.1 


13 


45.3 


14.4 


13 


II. Junior High Schoo 


1 Teache 


rs 










Video 


61.1 


14.5 


10 


72.9 


18.7 


10 


Wri tten 


67.2 


15.7 


12 


72.8 


13.3 


12 


Control 


58.8 


19.4 


11 

j 


55.5 


13.8 


11 
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Table 3 

Mean Number of Student Responses to Higher Cognitive Questions 





Precourse 


Posto 


ourse 


Group 


X 


S.D. 


J 


S.D. 


I . Grades 4-6 










Video 


32.3 


11.5 


38.9 


15.9 


Written 


31.1 


12.1 


47.1 


12.9 


Control 


38.2 


15.7 


31.4 


12.0 


II. Junior High School 










Video 


30.4 


6.8 


36.0 


16.3 


Wri tten 


39.0 


16.1 


55.3 


20.3 


Control 


29.6 


16.0 


34.0 


16.9 



Table 4 

Mean Number of Words per Student Response to 
Higher Cognitive Questions 



Group 


Pretape 


Posttape 


I. Grades 4-6 






Video 


15.0 


15.6 


Written 


11.3 


13.2 


Control 


11.5 


13.2 


II. Junior High School 






Video 


15.0 


18 9 


Written 


1».4 


17.8 


Control 


11.9 


13.2 



\ ^ 

! \ 
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Table 5 

Number of Students Giving Long Responses (15 words or more) 



Group 


Number of 
Classes 


Pretape 


Posttape 


I. Grades 4-6 








Video 


16 


164 


208 


Wri tten 


16 


133 


247 


Control 


13 


135 


121 


II. Junior High School 








Video 


10 


127 


175 


Wri tten 


12 


194 


250 


Control 

1 


11 


97 


106 

b 
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Table 6 ^ 

V 

Number of Probing Questions 
Precourse and Postcourse 



Group 


Preco 


Lirse 


Postcourse 


X 


S.D. 


J 


S.D. 


I. 4-6th Grade Teachers 










Video 


2.62 


2.24 


2.75 


1.83 


Wri tten 


1.93 


1.73 


4.81 


2.92 


Control 


2.38 


2.22 


2.46 


2.14 


11. Junior High School Teacher 


s 








Video 


2.10 


1.36 


2.20 . 


1.31 


Written 


1.58 


1.78 


2.83 


3.38 


Control 


2.90 


2.26 


1.63 


1.69 
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Table 7 

Frequency of Use of Refocusing Technique 
Precourse and Postcourse 





Precourse 


Postc 


ourse 


' X 


S.D. 


X 


S.D. 


I. 4-6th Grade Teachers 










Video 


3.84 


2.10 


3.62 


1.76 


Wri tten 


3.56 


1.84 


3.59 


1.40 


Control 


4.19 


2.43 


3.69 


1.84 


II. Junior High School Teacher 


s 








Video 


3.00 


1.83 


4.55 


2.63 


Wri tten 


2.58 


1.79 


3.87 


3.05 


Control 


3.63 


1.22 


4.59 


2.26 
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TABLE 8 

General Evaluation of M1n1course 9 





VIDEO (N>21) 

% 


WRITTEN (N=25) 
% 


1. As compared to other inserv* 
rate Mini course 9? 

Much better than 
Better than 
On a par with 
Worse than 
Much worse than 


Ice education experie 

52 

33 

15 

0 

0 


2 nces, how would you 

48 

44 

8 

0 

0 


2. As compared to your college 
would you rate Mini course 9 

Much better than 
Better than 
On a par with 
Worse than 
Much worse than 


(preservice) educat 

? 

35 

48 

17 

0 

0 


ion courses » how 

61 

20 

11 

8 

0 


3. To what extent has Minicour 
teaching? 

Considerable Improvement 
Some Improvement 
Slight Improvement 
No Improvement 


se 9 helped you impr 

8 

75 

13 

4 


! 

ove your overall 

23 

58 

16 

3 



